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Abstract. This paper studies a difference operator for stochastic systems whose 
specifications are represented by Abstract Probabilistic Automata (APAs). In the 
case refinement fails between two specifications, the target of this operator is to 
produce a specification APA that represents all witness PAs of this failure. Our 
contribution is an algorithm that allows to approximate the difference of two 
APAs with arbitrary precision. Our technique relies on new quantitative notions of 
distances between APAs used to assess convergence of the approximations as well 
as on an in-depth inspection of the refinement relation for APAs. The procedure is 
effective and not more complex to implement than refinement checking. 

1 Introduction 

Probabilistic automata as promoted by Segala and Lynch [38J are a widely-used for- 
malism for modeling systems with probabilistic behavior. These include randomized 
security and communication protocols, distributed systems, biological processes and 
many other applications. Probabilistic model checking [24, 5, 42J is then used to analyze 
and verify the behavior of such systems. Given the prevalence of applications of such 
systems, probabiUstic model checking is a field of great interest. However, and similarly 
to the situation for non-probabilistic model checking [10], probabilistic model checking 
suffers from state space explosion, which hinders its applicability considerably. 

One generally successful technique for combating state space explosion is the use of 
compositional techniques, where a (probabilistic) system is model checked by verifying 
its components one by one. This compositionality can be obtained by decomposition, 
that is, to check whether a given system satisfies a property, the system is automatically 
decomposed into components which are then verified. Several attempts at such automatic 
decomposition techniques have been made [12, 29], but in general, this approach has not 
been very successful [11]. 

As an altemative to the standard model checking approaches using logical specifica- 
tions, such as e.g. LTL, MITL or PCTL [34, 3, 21], automata-based specification theories 
have been proposed, such as Input/Output Automata [33], Interface Automata [13], and 
Modal Specifications [30, 35, 6]. These support composition at specification level; hence 
a model which naturally consists of a composition of several components can be verified 
by model checking each component on its own, against its own specification. The overall 
model will then automatically satisfy the composition of the component specifications. 
Remark that this solves the decomposition problem mentioned above; instead of trying 
to automatically decompose a system for verification, specification theories make it 
possible to verify the system without constructing it in the first place. 
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Moreover, specification theories naturally support stepwise refinement of specifica- 
tions, i.e. iterative implementation of specifications, and quotient, i.e. the synthesis of 
missing component specifications given an overall specification and a partial implemen- 
tation. Hence they allow both logical and compositional reasoning at the same time, 
which makes them well-suited for compositional verification. 

For probabilistic systems, such automata-based specification theories have been first 
introduced in [26], in the form of Interval Markov Chains. The focus there is only on 
refinement however; to be able to consider also composition and conjunction, we have 
in [8] proposed Constraint Markov Chains as a natural generalization which uses general 
constraints instead of intervals for next-state probabilities. 

In [15], we have extended this specification theory to probabilistic automata, which 
combine stochastic and non-deterministic behaviors. These Afcfracf Probabilistic Au- 
tomata (APA) combine modal specifications and constraint Markov chains. Our spec- 
ification theory using APA should be viewed as an alternative to classical PCTL [21], 
probabilistic I/O automata [32] and stochastic extensions of CSP [22]. Like these, its 
purpose is model checking of probabilistic properties, but unlike the alternatives, APA 
support compositionality at specification level. 

In the context of refinement of specifications, it is important that informative de- 
bugging information is given in case refinement fails. We hence need to be able to 
compare APA at the semantic level, i.e. to capture the difference between their sets of 
implementations. This is, then, what we attempt in this paper: given two APAs A^^i and 
N2, to generate another APA for which [iV] = [A^i] \ IN2I (where {N} denotes the 
set of implementations of N). 

As a second contribution, we introduce a notion of distance between APAs which 
measures how far away one APA is from refining a second one. This distance, adapted 
from our work in [40, 6], is accumulating and discounted, so that differences between 
APAs accumulate along executions, but in a way so that differences further in the future 
are discounted, i.e. have less influence on the result than had they occurred earlier. 

Both difference and distances are important tools to compare APAs which are not in 
refinement. During an iterative development process, one usually wishes to successively 
replace specifications by more refined ones, but due to external circumstances such as 
e.g. cost of implementation, it may happen that a specification needs to be replaced by 
one which is not a refinement of the old one. This is especially important when models 
incorporate quantitative information, such as for APAs; the reason for the failed refine- 
ment might simply be some changes in probability constraints due to e.g. measurement 
updates. In this case, it is important to assess precisely how much the new specification 
differs from the old one. Both the distance between the new and old specifications, as 
well as their precise difference, can aid in this assessment. 

Unfortunately, because APAs are finite-state structures, the difference between 
two APAs cannot always itself be represented by an APA. Instead of extending the 
formalism, we propose to approximate the difference. We introduce both over- and 
under-approximations of the difference of two APAs. We construct a sequence of under- 
approximations which converges to the exact difference, hence eventually capturing 
all PAs in |A^i] \ |A^2], and a fixed over-approximation which may capture also PAs 
which are not in the exact difference, but whose distance to the exact difference is zero: 
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hence any superfluous PAs which are captured by the over-approximation are infinitesi- 
mally close to the real difference. Taken together, these approximations hence solve the 
problem of assessing the precise difference between APAs in case of failing refinement. 

Related work. This paper embeds into a series of articles on APA as a specification 
theory [15-17]. In [15] we introduce deterministic APA, generalizing earlier work on 
interval-based abstractions of probabilistic systems [19, 26, 27|, and define notions of 
refinement, logical composition, and structural composition for them. We also introduce 
a notion of compositional abstraction for APA. In [16] we extend this setting to non- 
deterministic APA and give a notion of (lossy) determinization, and in [17] we introduce 
the tool APAC. The distance and difference we introduce in the present paper complement 
the refinement and abstraction from [15]. 

Compositional abstraction of APA is also considered in [39], but using a different 
refinement relation. Differences between specifications are developed in [36] for the 
formalism of modal transition systems, and distances between specifications, in the 
variant of weighted modal automata, have been considered in [6]. Distances between 
probabiUstic systems have been introduced in [14, 18, 41]. 

The originaUty of our present work is, then, the abiUty to measure how far away one 
probabilistic specification is from being a refinement of another, using distances and our 
new difference operator. Both are important in assessing precisely how much one APA 
differs from another. 

Acknowledgement. The authors wish to thank Joost-Pieter Katoen for interesting 
discussions and insightful comments on the subject of this work. 

2 Background 

Let Dist{S) denote the set of all discrete probability distributions over a finite set S and 

B2 = {T,±}. 

Definition 1. A probabilistic automaton (PA) [38] is a tuple {S, A, L, AP, V, sq), where 
S is a finite set of states with the initial state sq & S, A is a finite set of actions, L: 
S X Ax Dist{S) B2 is a (two-valued) transition function, AP is a finite set of atomic 
propositions and V: S 2"^^ is a state-labeling function. 

Consider a state s, an action a, and a probability distribution /i. The value of L{s, a, p) 
is set to T in case there exists a transition from s under action a to a distribution /i on 
successor states. In other cases, we have L(s, a, ji) = _L. We now introduce Abstract 
Probabilistic Automata (APA) [15], that is a specification theory for PAs. For a finite 
set S, we let C {S) denote the set of constraints over discrete probability distributions 
on S. Each element G C{S) describes a set of distributions: Sat{ip) C Dist{S). Let 
B3 = {T, ?, _L}. APAs are formally defined as follows. 

Definition 2. An APA [15] is a tuple {S, A, L, AP, V, Sq), where S is a finite set of 

states, Sq Q S is a set of initial states, A is a finite set of actions, and AP is a finite set of 
atomic propositions. L : S x A x C{S) — >■ B3 is a three-valued distribution-constraint 
function, and V : S^2^ maps each state in S toa set of admissible labelings. 
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APAs play the role of specifications in our framework. An APA transition abstracts 
transitions of a certain unknown PA, called its implementation. Given a state s, an action 
a, and a constraint (p, the value of L(s, a, 95) gives the modality of the transition. More 
precisely, the value T means that transitions under a must exist in the PA to some 
distribution in Sat{(p); ? means that these transitions are allowed to exist; _L means 
that such transitions must not exist. We will sometimes view i as a partial function, 
with the convention that a lack of value for a given argument is equivalent to the _L 
value. The function V labels each state with a subset of the powerset of AP, which 
models a disjunctive choice of possible combinations of atomic propositions. We say that 
an APA N = (S, A, L, AP, V, Sq) is in Single Valuation Normal Form (SVNF) if the 
valuation function V assigns at most one valuation to all states, i.e. V.s G S, \V{.s)\ < 1. 
From 1 15], we know that every APA can be turned into an APA in SVNF with the same 
set of implementations. An APA is deterministic [15] if (1) there is at most one outgoing 
transition for each action in all states, (2) two states with overlapping atomic propositions 
can never be reached with the same transition, and (3) there is only one initial state. 

Note that every PA is an APA in SVNF where all constraints represent single-point 
distributions. As a consequence, all the definitions we present for APAs in the following 
can be directly extended to PAs. 

Let N = {s, A, L, AP, V, {sq}) be an APA in SVNF and let v C AP. Given a state 
s G S and an action a & A, we will use the notation sucCs_a(t') to represent the set of 
potential o-successors of s that have v as their valuation. Formally, sucCs a(w) = {s' € 
S I V{s') = {v}, 3^ e C(S'), G Sat{ip) : L{s, a, (f) ^ _L, Ai(s') > 0}. When clear 
from the context, we may use sucCs, a(s') instead of sucCs,a(y(s')). Remark that when 
is deterministic, we have |sucCs,a(t^)| < 1 for all s, a, v. 

3 Refinement and Distances between APAs 

We introduce the notion of refinement between APAs. Roughly speaking, refinement 
guarantees that if Ai refines A2, then the set of implementations of Ai is included in the 
one of A2. We first recall the notion of simulation between two given distributions. 

Definition 3 ([15]). Let S and S' be non-empty sets, and /it, /z' be distributions; fi G 
Dist{S) and /x' £ Dist{S'). We say that fi is simulated by fi' with respect to a relation 
TZ C S X S' and a correspondence function S : S ^ {S' ^ [0, 1]) iff 

1. for all s G S with /i(s) > 0, S{s) is a distribution on S', 

2. /oralis' G S', EsgsM(s) • S{s){s') = fi'{s'), and 

3. whenever 5(s){s') > 0, then (s, s') G TZ. 

We write ^ fi' if ji is simulated by jj.' w.r.t TZ and 5, and p (^n m' if there exists 5 
with p h'. 

We will also need distribution simulations without the requirement of a relation 
TZ C S X S' (hence also without claim 3 above); these we denote by ^ /i' . 

Definition 4 ([15]). Let Ni = {Si, A, Li,AP, Vi,Sl^) andN2 = {S2,A, L2.AP. V2, S^) 
be APAs. A relation TZ C Si x S2 is a refinement relation if and only if, for all 
(si, S2) e TZ, we have Vi{si) C ¥2(32) and 
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1. Va e A, V(p2 e C{S2), ifL2{s2, a, (^2) = T, then 3ipi e C(5i) : ii(si, a. ipi) = 
T ant/ V/ii € 5ai((/3i), 3/X2 G Sat{(p2) such that fii (57^ /i27 

2. Va e ^, V(pi € C{Si), if Li{si,a,ipi) 7^ _L, f/jen 3lp2 G C(<S'2) ■swc/j 
L2(s2, a, LP2) 7^ -L andy^i G Sat{(pi), 3^2 G Sat(ip2) such that ni (s-jz ^2- 

We say that A'^i refines N2, denoted Ni ^ N2, iff there exists a refinement relation 
such that VsJ e S^, e 5^ : (.S(^, sg) e 7?.. Since any PA P is also an APA, we 
say that P satisfies N (or equivalently P implements N), denoted P \= N,iff P ^ N. 
In [15], it is shown that for deterministic APAs A^i, A'2, we have A^i < N2 '^=> 
I-^il ^ 1^2], where (Nij denotes the set of implementations of APA Ni. Hence 
for deterministic APAs, the difference |A^i] \ |A^2] is non-empty iff Ni N2- This 
equivalence breaks for non-deterministic APAs [15], whence we develop our theory only 
for deterministic APAs. 

To show a convergence theorem about our difference construction in Sect. 4.2 
below, we need a relaxed notion of refinement which takes into account that APAs 
are a quantitative formalism. Indeed, refinement as of Def. 4 is a purely qualitative 
relation; if both A^2 Ni and N\, then there are no criteria to compare N2 

and A^s with respect to A^i, saying which one is the closest to N\. We such a relaxed 
notion by generalizing refinement to a distance which provides precisely such criteria. 
In Sect. 4.2, we will show how those distances can be used to show that increasingly 
precise difference approximations between APAs converge to the real difference. The 
next definition shows how a distance between states is lifted to a distance between 
constraints. 

Definition 5. Let d : Si x S2 ^ K"*" and ipi e C{Si), ip2 € 0(82) be constraints in 
Ni and N2. Define the distance D^^^n^ between and (p2 as follows: 

DNi,N2{'fil,'P2,d) = 



For the definition of d below, we say that states si e Si, S2 e 52 are not compatible 
if either (1) Vi{si) ^ 1^2 (•''■2), (2) there exists a e A and e C'{Si) such that 
Li{si,a, fi) ^ ± and for all ip2 <= C(52), ^2(52, a, V2) = -L, or (3) there exists a G A 
and (p2 € C{S2) such that .^2(52, 0',ip2) = T and for all ipi G C{Si),Li{si,a, ipi) ^ 
T. For compatible states, their distance is similar to the accumulating branching distance 
on modal transition systems as introduced in [6,40], adapted to our formalism. In the 
rest of the paper, the real constant < A < 1 represents a discount factor. Formally, 
: 5i X 52 — *• [0, 1] is the least fixpoint to the following system of equations: 



sup inf I inf 



} , /il(si)(5(si,S2)d(si,S2) 

(si,S2)eS'lXS2 



d{si,S2) = 



(1) 




1 if si is not compatible with S2 



max min 

{0,(^2:^2(82, a, ¥'2)=T}{¥'i:Li(si, a, v3i)=T} 



max min 

{a,¥'i:Li(si,o,v3i)7^-L}{V32:i'2(s2,a,v32)#-L} 



XDN^^N^{(fil,ip2,d) 
Ar'jVi,JV2(<^l,<^2,rf) 



Otherwise 
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Since the above system of linear equations defines a contraction, the existence and 
uniqueness of its least fixpoint is ensured, cf. [31]. This definition intuitively extends to 
PAs, which allows us to propose the two following notions of distance: 

Definition 6. Let Ni = {Si, A, Li,AP, Vi,Sl) and N2 = {82, A, L2, AP, V2, S^) be 
APAs in SVNF. The syntactic distance and thorough distances between N\ and N2 are 
defined as follows: 

- Syntactic distance. d{Ni,N2) = max^iggi (min^2g5:2 d{sl, sg)). 

- Thorough distance. dt{Ni,N2) = supp^gj^Vi] ( infpseiJVsi ^(-Pi, ^2)). 

Note that the notion of thorough distance defined above intuitively extends to sets of 
PAs: given two sets of PAs Si, §2, we have (i( (81,82) = supp^gg^ (infp^eSs d{PijP2))- 

The intuition here is that d{si , S2) compares not only the probability distributions 
at Si and S2, but also (recursively) the distributions at all states reachable from ,si and 
S2, weighted by their probability. Each step is discounted by A, hence steps further 
in the future contribute less to the distance. We also remark that A^i ^ N2 implies 
d{N'i, N'2) = 0. It can easily be shown, cf. [40], that both d and dt are asymmetric 
pseudometrics (or hemimetrics), i.e. that they satisfy d{Ni,N\) = and d{Ni,N2) + 
d{N2,Nz) > d{Ni,N-i) for all APAs iVi, A^2, -/V3 (and similarly for d*). The fact that 
they are only pseudometrics, i.e. that d(Ni, N2) = does not imply Ni = N2, will 
play a role in our convergence arguments later. The following proposition shows that 
the thorough distance is bounded above by the syntactic distance. Hence we can boimd 
distances between (sets of) implementations by the syntactic distance between their 
specifications. 

Proposition!. For all APAs Ni and N2 in SVNF, it holds that dt{Ni,N2) < d{Ni,N2). 
4 Difference Operators for APAs 

The difference A^i \ N2 of two APAs A^i , N2 is meant to be a syntactic representation of 
all counterexamples, that is, all PAs P for which P e [A^i] but P ^ IN2}. We will see 
later that such difference cannot be an APA itself; instead we will approximate it using 
APAs. 

Because A^i and N2 are deterministic, we know that the difference [iVi] \ \N2\ 
is non-empty if and only if Ni N2. So let us assume that Ni 2< A^2, and let TZ be 
a maximal refinement relation between Ni and N2. Since Ni N2, we know that 
(sq, Sq) ^ TZ. Given (si, S2) € 5*1 x S2, we can distinguish between the following cases: 



1. {si,s2)en 

2. Vi(si) ^1/2(52), 

3. (si,S2) and Vi(si) = 1^2(52), and 



(a) there exists e £ A and ipi € C{Si) such that 
Li{si,e,<pi) = TandV(^2 e €{82) : -^2(52, e, ^52) = -L, 



p 



e,T 



e 



o 
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(b) there exists e G A and (pi € C{Si) such that 
e, (fii) =? and V(p2 e C(S'2) : ^2(52,6, </?2) = -L, 



(c) there exists e G A and </ji e C(S'i) such that 

Li(si,e, >? and 3(^2 € C(S'2) : ^2(52, e, (/32) = 
?,3/i € Sat{ip\) such that V/Lt' G Sat{(f2) '■ A* 07?. Z^'- 

(d) there exists e e A and (p2 € 0(82) such that 
L2(s2,e,<^2) = T andV(^i e C(5i) : Li(si,e,(^i) = ±, 



P 9 



T t 

VI 

<=,{?, T} 

e,? 

T T 



Sl 





S2 

o 



(e) there exists e G A and (/52 G C(S'2) such that 
L2(s2, e, <y92) = T and 3(^i e C{Si) : Li(si, e, (^1) =?, 



(f) there exists e S A and (f2 S C'(5'2) such that 
L2(s2, 6,(^2) = T, 3(^1 e C(6'i) : Li(si, e, </ji) = T 
and 3/U e Sat{(pi) such that V/u' e Sat{ip2) ■ /U /u'. 



Sl 

p 



T 



S2 

o 



e,T 



S2 

o 



t t 



Remark that because of the determinism and SVNF of APAs Ni and N2, cases 1, 2 
and 3 cannot happen at the same time. Moreover, although the cases in 3 can happen 
simultaneously, they cannot be "triggered" by the same action. In order to keep track of 
these "concurrent" situations, we define the following sets. 

Given a pair of states (si, S2), let us define Ba{si, S2) to be the set of actions in 
A such that case 3. a above holds. If there is no such action, then Ba(si,S2) = 0. 
Similarly, we define Bb{si,S2), i?c(si, S2), i3d(si, S2), Be(si, S2) and i?/(si, S2) to be 
the sets of actions such that cases 3.6, c, d, e and 3./ holds respectively. Given a set 
X C {a,b,c,d,e,f}, M Bx{si,S2) = Ua;ex-Bx(si, S2). In addition, let5(si,S2) = 

B{a,b,c,d,e,f}{si,S2)- 

4.1 Over-Approximating Difference 

We now try to compute an APA that represents the difference between the sets of 
implementations of two APAs. We first observe that such a set may not be representable 
by an APA, then we will propose over- and under-approximations. Consider the APAs 
A^i and N2 given in Figures la and lb, where a ^ /3 ^ 7. Consider the difference of 
their sets of implementations. It is easy to see that this set contains all the PAs that can 
finitely loop on valuation a and then move into a state with valuation /?. Since there is 
no bound on the time spent in the loop, there is no finite-state APA that can represent 
this set of implementations. 
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KJJ^ \1J (Ml) = 1) V (p(2) = 1) = 1) V (MB) = 1) 



(a) APA iVi (b) APA N2 

Fig. 1: APAs Ni and A^2 such that |iVi| \ IN2} cannot be represented using a finite-state 
APA. 



Now we propose a construction \* that over-approximates the difference between 
APAs in the following sense: given two deterministic APAs Ni = {Si, A, Li, AP, 
Vi,{sl}) and N2 = (5*2, A, L2, AP, V2, {sD) in SVNF, such that TVi N2, we have 
{Nil \ [7V2I C |iVi V iV2l. We first observe that if Viisl) + ^2(5^), i.e. (sj, sg) in 
case 2, then lATi] n |7V2] = 0. In such case, we define iVi \* N2 as A^i. Otherwise, 
we build on the reasons for which refinement fails between iVi and A^2- Note that the 
assumption that TVi N2 implies that the pair (sj, Sq) can never be in any refinement 
relation, hence in case 1 . We first give an informal intuition of how the construction 
works and then define it formally. 

In our constraction, states in Ni \* N2 will be elements of Si x (5'2U{_L}) x (^U{e}). 
Our objective is to ensure that any implementation of our constructed APA will satisfy 
A^i and not N2. In (si, S2, e), states si and S2 keep track of executions of Ni and A^2. 
Action e is the action of A^'i that will be used to break satisfaction with respect to N2, 
i.e. the action that will be the cause for which any implementation of (si, S2, e) cannot 
satisfy N2. Since satisfaction is defined recursively, the breaking is not necessarily 
immediate and can be postponed to successors. _L is used to represent states that can 
only be reached after breaking the satisfaction relation to N2- In these states, we do not 
need to keep track of the corresponding execution in A^2. thus only focus on satisfying 
A''i. States of the form (si, S2, e) with S2 9^ -L are states where the satisfaction is broken 
by a distribution that does not match constraints in N2 (cases 3.c and 3.f). In order to 
invalidate these constraints, we still need to keep track of the corresponding execution in 
N2, hence the use of e instead of _L. 

The transitions in our construction will match the different cases shown in the 
previous section, ensuring that in each state, either the relation is broken immediately or 
reported to at least one successor. Since there can be several ways of breaking the relation 
in state (sj, Sg), each corresponding to an action e € i3(sj, Sq), the APA A^i \* N2 will 
have one initial state for each of them. Formally, if (sj, Sg) is in case 3, we define the 
over-approximation of the difference of Ni and N2 as follows. 

Definition 7. Let Ni \* N2 = {S, A, L, AP, V, So), where 5 = x (52 U {_L}) x 
(AU{£}), V{si,S2,a) = V{si)forall S2anda, So = {{so,slJ) \ f G B(sJ,sg)}, 
and L is defined by: 

- Ifs2 = J- or e = e or {si, S2) in case 1 or % then for all a G A and ip S C{Si) such 
that Li(si,a,ip) ^ _L, let L{{si,S2,e),a,(p-^) = Li{si,a,ip), with (p-^ defined 
below. For all other h ^ A and (p G C{S), let L{{si, S2, e), b, (p) = _L. 

- Else, we have (si, S2) in case 3 and B{si, S2) ^ 9 by construction. The definition 
ofL is given in Table 1, with the constraints ip^ and defined hereafter. 
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Ni,N 


2 


A^i \* N2 


Formal Definition of L 


-fc>a(Sl, S2) 


I \ 


1^ 


(»i. 


) 

± 
1 


For all a 7^ e G ^ and tp G C'(S'i) such 
that Li{s\,a,ip) ^ ±, let L((si, S2, e), a, (^^) = 
Li(si, a, (/p). In addition, let L((si, S2, e), e, (pj^) = 
T. For all other h G ^ and £ C{S), let 
L((si,S2,e),6, f^) = ±. 


Bt,{si,S2) 




1^ 


Bd{si, S2) 


\ 


1 ^2 

■ 4 




2.'-) 

5 


For all a G A and G C{Si) such that Li (si ,a,ip) ^ ±, 
let I/((si, S2, e), a, = L\(a\, a, ip). For all other 6 G 
^and(/3 G C(S'), let L((si, S2, e), 6, (p) = ±. 


B^•is^ . 59) 




1 


9 

e,? 

T 


For all a e £ A and (^9 G C'(S'i) such 
that Li{si,a,(p) ^ ±, let I/((si, S2, e), a, y^) = 

-L/ j_ 1 j_ , J 1 • ill duuiuuii, ivi 1-/ iioij /' ' r 12 / — ' * 

all other 6 G A and <p G C(5'), let L((si, S2, e), 6, 93) = 
±. 


-Bc(si,S2) 


i 


■'2 

e,{?,T) 


^.y \e,{?,T} 

912 <fi 


For all a G A £ind fi G C(5i) such that I/i(si, a, (ys) 7^ 
_L (including e and p\), let L((si, S2, e), a, = 
Li(si,a, In addition, let L((si, S2, e), e, <^f2) = 
T. For all other h G A and </? G C(S'), let 
i((si,S2,e),6, <p) = ±. 


B/(S1,S2) 


i 


-1 

1 7^ V 


1 

2 



Table 1: Definition of the transition function LinNi\* N2. 



Given (p G C{Si), ip-^ G C(5) is defined as follows: fi G Sat{(p-^) iff Vsi € 5i,Vs2 
_L,V6 ^ e, n{si,S2, b) = and the distribution (/x 4.1: si /u(si, _L, e)) is in Sat{ip). 
Given a state (si, S2,e) G S with S2 ^ -L and e ^ e and two constraints </3i G C(5i), 
</32 € C('S'2) such that Li(.si, e, lySi) 1. and ^2(52, e, (/32) 7^ -L, the constraint ipf2 S 
(7(5*) is defined as follows; /z G S'at((^f2) iff (1) for all {s'i,s'2,c) G 5, we have 
/i(s'i, S2, c) > => s'2 = -L if sucCs2,e{s[) = and s'2 = sucCs2,e(s'i) otherwise, and 
c G B(s'i, s'2) U {e}, (2) the distribution ^1 : s[ ^ EcgAu{£},4gS2U{±} ^2, c) 
satisfies (fi, and (3) either (a) there exists {s[,-L, c) such that iJ,{s[,-L, c) > or (b) 
the distribution /Lt2 : S2 X]ceAu{e} s{eSi /"(s'd does not satisfy <^2, or (c) there 
exists s[ G 5*1, .S2 G S'2 and c 7^ £ such that ^{s[,S2,c} > 0. Informally, distributions 
in (pf2 must (1) follow the corresponding execution is A^i and N2 if possible, (2) satisfy 
ipi and (3) either (a) reach a state in A^i that cannot be matched in N2 or (b) break the 
constraint (p2, or (c) report breaking the relation to at least one successor state. 

The following theorem shows that A^i \* N2 is an over-approximation of the differ- 
ence of Ni and N2 in terms of sets of implementations. 

Theorem 1. For all deterministic APAs Ni and N2 in SVNF such that Ni ^ N2, we 
have [Nil \ |iV2] C {Ni \* N2I 
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a,l 




H e Satlipf^) {ii{l,A,a) + ii{l,A,e) = 1) A (/j(l,^,o) > 0) 

V{m(2,±,£) = 1) 



(a) A^i \* N2 



(b) P 



Fig. 2: Over-approximating difference A^i \* N2 of APAs A^i and N2 from Figure I and 
PA P such that P ^ A^i \* 7V2 and P ^ N2. 

The reverse inclusion in the above theorem does not hold. Intuitively, as explained 
in the construction of the constraint (pfg above, one can postpone the breaking of 
the satisfaction relation for N2 to the next state (condition (3.c)). This assumption is 
necessary in order to produce an APA representing all counterexamples. However, when 
there are cycles in the execution of A^i \* this assumption allows to postpone forever, 
thus allowing for implementations that will ultimately satisfy N2. This is illustrated in 
the following example. 

Example 1. Consider the APAs A^^i and N2 given in Fig. 1. Their over-approximating 
difference Ni\* N2 is given in Fig. 2a. One can see that the PA P in Fig. 2b satisfies 
both Ni\* N2 and iVa. 

We will later see in Corollary 1 that even though Ni\* N2 may be capturing too many 
counterexamples, the distance between A^i \* A^2 and the real set of coimterexamples 
|7Vi] \ |A^2l is zero. This means that the two sets are infinitesimally close to each other, 
so in this sense, Ni \* N2 is the best possible over-approximation. 

4.2 Under- Approximating Difference 

We now propose a construction that instead under-estimates the difference between 
APAs. This construction resembles the over- approximation presented in the previous 
section, the main difference being that in the imder-approximation, states are indexed 
with an integer that represents the maximal depth of the unfolding of counterexamples. 
The construction is as follows. 

LetiVi = {SuA,LuAP,Vi,{sl}) and TVs = {S2,A,L2,AP,V2A4}) be two 
deterministic APAs in SVNF such that A^i ^ N2- Let i^T € N be the parameter of 
our construction. As in Section 4.1, if Vi{sq) ^ V2{sq), i.e. (sj, Sq) in case 2, then 
|7Vi] n |iV2] = 0. In this case, we define A^i N2 as Ni. Otherwise, the under- 
approximation is defined as follows. 

Definition 8. Let Ni N2 = {S, A, L, AP, V, S^), where = S'l x (S'2 U {_L}) x 
{A\J{£]) X {l,...,K}, V{si,S2,a,k) = V{si) for all S2, a, k < K, = 
{{sl,SQ,f,K) I / e B{sl, Sq)}, and L is defined by: 
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e € 






2 


A^i N2 


Formal Definition of L 


-fc>a(Sl, S2) 


}■ \ 


)^ 


(»1.»2.1'.I0 

-I 


For all a / e e A and € C'(S'i) such 
that Li{si,a,(p) ^ -L, let L{{si,S2,e,k),a,(p''-) = 
Li(si,a,(p). In addition, let L({si,S2,e,k),e,ipi ) = 
T. For all other b G A and ip € C{S), let 
i((si,S2,e,A:),6, (p) = ±. 








Bd{si, S2) 


\ 


1 s 

■ 


1 

2 


(Sl.»2.<'.//) 


For all a 6 A and ip 6 C(S'i) such that Li {si ,a,ip) ±, 
l&t L{{si,S2,e,k),a,ip''-) = Li(si, a, t/?). For all other 
6 e ^ and 99 e C{S), let L((si, S2, e, k), b, tp) = ±. 






1 

2 


(#i,,s^,e,fc) 

9 

e,? ; 
T 


For all a 7^ e £ ^ and tp G C{Si) such 
that I/i(si, a, / ±, let I/((si, S2, e, fe), a, i/?^) = 
Li(,s'i, a, ip). In addition, let L((si, S2, e, A;), e, v'i'2''°) = 
?. For all other b G ^ and 95 G C{S), let 
i((si,S2,e,fe),6, (p) = ±. 


Bc{si,S2) 


\ 


1 32 

> 

e,{?,T) 


(.i,,,2,e,fc) 

e.y y,{?,T} 


For all a G A and G C{Si) such that Z/i(si, a, (p) 7^ 
-L (including e and t^i), let L((si, S2, e, fc), 0, (p"'") = 
Li(si, 0, 1^). In addition, let L((si, S2, e, fe), e, <pf2''°) = 
T. For all other b € A and ip € C{S), let 
L{{si,S2,e,k),b, ip) = ±. 


Bf{si,S2) 


\ 


' -1 

1 7^ V 


1 

2 



Table 2: Definition of the transition function L in Ni\^ N2. 



— If S2 = \- or e = e or {s\. S2) in case 1 or 2, then for all a £ A and <p> G C(S'i) 
such that Li{si,a, (p) 7^ _L, let L{{si, S2, e, fc), o, ip-^) = Li{si,a, (p), with (p-^ 
defined below. For all other h €: A and ip> € C{S), let L{{si, S2, e, k), b, ip) = _L. 

- Else we have (si, S2) in case 3 and B{si, S2) ^ by construction. The definition of 
L is given in Table 2. The constraints (p-^ and (pi2 are defined hereafter 

Given a constraint ip G C{Si), the constraint (p-^ e C{S) is defined as follows: 
fl e Sat{ip-^) iffVsi e S'i,Vs2 -L,V6 ^ e,Vfc ^ 1, iJ,{si,S2,b,k) = and the 
distribution (/x |i: si i-> /z(si, _L, £, 1)) is in S'af((/5). Given a state (si, S2, e, /e) € S 
with S2 7^ -L and e ^ e and two constraints ipi G C{Si) and 932 € C(S'2) such that 
Li(si, e, 951) ^ _L and ^2(52, e, (/32) 7^ -L, the constraint <pf2'^ G C'(S') is defined as 
follows: fl G Sat{(pf2'^) iff (1) for all {s'i,s'2, c, k') G S, if /Lt(si , S2, c, /c') > 0, then c G 
S2) U {e} and either sucCs2,e(s'i) = 0. ^2 = -L and k' = 1, or s'2 = sucCs2,eis[), 
(2) the distribution fii : s[ ^ EceAu{e},4GS2u{±},fe'>i /^(^'i' 4> c, A:') satisfies ipi, 
and (3) either (a) there exists {s[, J-,c,l) such that fj,{s[, J-,c,l) > , or (b) the 
distribution jU2 : S2 J2cGAu{E},s[GSi,k'>i M('Si, S2: c, fc') does not satisfy ip2, or (c) 
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B,2 



{{a}}(l,±,.X 



kJ am 

fi e Sat(ip^i^) ^ (/j(2,±,£,l) = l) 



/< e Sat{ipff) A, a, 2) + ^, o, 1) + A, £, 1) = 1) 

A(/i(l, A.a. 1) > 0) 
VW2,±,£,1) = 1) 



(a) ATi \i iVa 



(b) iVi \2 JV2 



Fig. 3: Under-approximations at level 1 and 2 of the difference of APAs Ni and A^2 from 
Figure 1. 

fc ^ 1 and ttiere exists s'l e 5*1, G ^2, c ^ £ and k' < k such that ii{s'i,s'2, c, k') > 0. 
The construction is illustrated in Figure 3. 

4.3 Properties 

We already saw in Theorem 1 that A^i \* A^2 is a correct over-approximation of the 
difference of A^i by A^2 in terms of sets of implementations. The next theorem shows 
that, similarly, all Ni\^ N2 are correct under-approximations. Moreover, for increasing 
K the approximation is improving, and eventually all PAs in |A^i| \ IA'^2] are getting 
caught. (Hence in a set-theoretic sense, lim/f^od^^i \''^ ^2] = I^il \ 1^2]-) 

Theorem 2. For all deterministic APAs Ni and N2 in SVNF such that Ni y< N2: 

1. for all KGN,we have Ni \^ N2 ^ Ni \^+^ N2, 

2. for all K eN, {Ni \^ N2I C |iVi] \ IN2I and 

3. for all PAP e |7Vi] \ IN2I there exists e N such that P G |A^i \^ N2I. 

Note that item 3 imphes that for all PA P e |A^i] \ |A'^2l, there is a finite specification 
capturing [A^il \ [iVa] "up to" P. 

Using our distance defined in Section 3, we can make the above convergence result 
more precise. The next proposition shows that the speed of convergence is exponential 
in K; hence in practice, K will typically not need to be very large. 

Proposition 2. Let N\ and N2 be two deterministic APAs in SVNF such that Ni N2, 
and let Ke N. Then dt{lN^l \ [A^aJ, [A^i ^^2!) < A^(l - A)-i. 

For the actual application at hand however, the particular accumulating distance d 
we have introduced in Section 3 may have hmited interest, especially considering that 
one has to choose a discounting factor for actually calculating it. 

What is more interesting are results of a topological nature which abstract away from 
the particular distance used and apply to all distances which are topologically equivalent 
to d. The results we present below are of this nature. 
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It can be shown, c.f. [40], that accumulating distances for different choices of A 
are topologically equivalent (indeed, even Lipschitz equivalent), hence the particular 
choice of discounting factor is not important. Also some other system distances are 
Lipschitz equivalent to the accumulating one, in particular the so-called point-wise and 
maximum-lead ones, see again [40]. 

Theorem 3. Let Ni and N2 be two deterministic APAs in SVNF such that Ni N^. 

1. The sequence {Ni \^ N2)KeN converges in the distance d, and limx^-oo d{Ni \* 

iV2,A^i\'^iV2) =0. 

2. The sequence (|A^i\''^-/V2])i<:eN converges in the distance dt, andlim.K->-oo dt{lNil\ 
IN2I [ATi \^ N2I) = 0. 

Recall that as d and dt are not metrics, but only (asymmetric) pseudometrics 
(i.e. hemi-metrics), the above sequences may have more than one limit; hence the 
particular formulation. The theorem's statements are topological as they only allure 
to convergence of sequences and distance 0; topologically equivalent distances obey 
precisely the property of having the same convergence behaviour and the same kernel, 
c.f. [1]. 

The next corollary, which is easily proven from the above theorem by noticing 
that its first part implies that also lim^^oo dtHNi \* N2j, {Ni \^ N2j) = 0, shows 
what we mentioned already at the end of Section 4.1: Ni \* N2 is the best possible 
over-approximation of |A''i] \ IN2I. 

Corollary 1. Let Ni and N2 be two deterministic APAs in SVNF such that Ni N2. 
Then dtilN^ \* N2I [A^i] \ [TVaD = 0. 

Again, as dt is not a metric, the distance being zero does not imply that the sets 

|A''i \* iV2] and |A^i] \ [A^2l are equal; it merely means that they are indistinguishable 
by the distance dt, or infinitesimally close to each other. 

5 Conclusion 

We have in this paper added an important aspect to the specification theory of Abstract 
Probabilistic Automata, in that we have shown how to exhaustively characterize the 
difference between two specifications. In a stepwise refinement methodology, difference 
is an important tool to gauge refinement failures. 

We have also introduced a notion of distance between specifications which can 
be used as another measure for how far one specification is from being a refinement 
of another. Using this distance, we were able to show that our sequence of under- 
approximations converges, semantically, to the real difference of sets of implementations, 
and that our over-approximation is infinitesimally close to the real difference. 

There are many different ways to measure distances between implementations and 
specifications, and in this paper we have chosen one specific. Apart from the fact that 
this can indeed be a useful distance in practice, we remark that the convergence results 
about our under- and over-approximations are topological in nature and hence apply 
with respect to all distances which are topologically equivalent to the specific one used 
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here. We plan to elaborate on the notion of distances between APAs in a future paper, 
where we will treat several different distances and work out their relation. 

We also remark that we have shown that it is not more difficult to compute the 
difference of two APAs than to check for their refinement. Hence if a refinement failure 
is detected (using e.g. the methods presented in our APAC tool), it is not difficult to also 
compute the difference for information about the reason for refinement failure. 

One limitation of our approach is the use of deterministic APAs. Even though 
deterministic specifications are generally considered to suffice from a modeling point 
of view [30], non-determinism may be introduced e.g. when composing specifications. 
Indeed, our constructions themselves introduce non-determinism: for deterministic APAs 
A''i, N2, both A^i \* A^2 and A^i \^ A'2 may be non-deterministic. Hence it is of interest 
to extend our approach to non-deterministic specifications. The problem here is, however, 
that for non-deterministic specifications, the relation between refinement and inclusion 
of sets of implementations A^i ^ N2 |A^i] C |A'2| breaks: we may well have 
Ni y< N2 but |iVi] C |iV2], cf. [15]. So the technique we have used in this paper to 
compute differences will not work for non-deterministic APAs, and techniques based on 
thorough refinement will have to be used. We will attempt to do this in a future paper. 

As a last note, we wish to compare our approach of difference between APA specifica- 
tions with the use of counterexamples in probabilistic model checking. Counterexample 
generation is studied in a number of papers [2, 20, 43, 4, 25, 37, 23, 44, 9, 28], typically 
with the purpose of embedding it into a procedure of counterexample guided abstraction 
refinement (CEGAR). The focus typically is on generation of one particular counterex- 
ample to refinement, which can then be used to adapt the abstraction accordingly. 

In contrast, our approach at computing APA difference generates a representation 
of all counterexamples. Our focus is not on refinement of abstractions at system level, 
using coimterexamples, but on assessment of specifications. This is, then, the reason 
why we want to compute all counterexamples instead of only one. We remark, however, 
that our approach also can be used, in a quite simplified version, to generate only one 
counterexample; details of this are in the appendix. Our work is hence supplementary 
and orthogonal to the CEGAR-type use of counterexamples: CEGAR procedures can 
be used also to refine APA specifications, but only our difference can assess the precise 
distinction between specifications. 
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Appendix: Counter-Example Generation 

Here we show how some techniques similar to the ones we have introduced can be used 
to generate one counterexample to a failed refinement A^^i ;^ Note that when we 
compute the approximating differences A^i \* N2 and Nx \^ N2, we are in principle 
generating (approximations to) the set of all counterexamples, hence what we do in 
Section 4 is much more general than what we will present below. Generating only one 
counterexample may still be interesting however, as it is somewhat easier than computing 
the differences iVi \* -^1 \^ N2 and is all that is needed e.g. in a CEGAR approach. 

First remark that Definition 4 can be trivially turned into an algorithm for checking 
refinement. Let iVi = (Si, A, Li,AP, Vi, {si}) and N2 = {S2,A, L2,AP, V2, {si}) 
be two deterministic APAs in S VNF. Consider the initial relation TZq = S1XS2. Compute 
TZk+i by removing all pairs of states not satisfying Definition 4 for TZk- The sequence 
(^n)rieN is then strictly decreasing and converges to a fixpoint within a finite number of 
steps K < \Si X 52 1 . This fixpoint TZk coincides with the maximal refinement relation 
TZ between Ni and N2. Let the index of this fixpoint be denoted with lnd(7^) = K; 
hence \ndTz{si,S2) = min(max({fc | (si, S2) e T^k}),K). 

We now observe that if a pair of states (si, S2) is removed from the relation TZ by 
case 3, then we need to keep track of the actions that lead to this removal in order to use 
them in our counterexample. Whenever a pair of states is in cases 3. a, 3.b, 3.d or 3.e, we 
have that Ind7^(si, S2) = and the counterexample can be easily produced by allowing 
or disallowing the corresponding transitions from Ni and A^2- Cases 3.c and 3.f play a 
different role: due to the fact that they exploit distributions, they are the only cases in 
which refinement can be broken by using its recursive axiom. In these cases, producing 
a counterexample can be done in two ways: either by using a distribution that does not 
satisfy the constraints in N'2 (if such a distribution exists, then \ndTi{si, S2) ~ 0), or by 
using a distribution that reaches a pair of states {s[, s'2) ^ TZ. When < lnd-R,(si, S2) < 
lnd(7?.), only the latter is possible. This recursive construction has disadvantages: it 
allows us to produce loops that may lead to incorrect counterexamples. In order to 
prevent these loops, we propose to use only those distributions that decrease the value of 
Ind in this particular case. The set Break(si , S2) defined hereafter allows us to distinguish 
the actions for which the value of Ind decreases, hence ensuring (by Lemma 1 below) 
the correctness of om counterexample construction. Let (si, S2) € Si x S'2 be such 
that Vi(si) C ¥2(82) and \rtdn{si,S2) = k < lnd(7^). We define Break(si,S2) to 
be the set {a S A | either a G Ba,b,d.e{si, S2) or there exists ipi € C{Si) such that 
Li{si,a, ipi) _L, </?2 € C{S2) such that L2{s2,a, (^2) 7^ -L and /Ui e Sat{ip\) such 
thatV/i2 e Sat{(p2), Mi '0nk ^2}- 

Remark that the conditions defined above are exactly the conditions for removing a 
pair of states (si, S2) at step k of the algorithm for computing TZ defined above. Under 
the assvunption that Fi(si) C V2(s2) and Ind7^(si, S2) — k < lnd(7?.), we can be sure 
that the set Break(si, S2) is not empty. Moreover, we have the following lemma. 

Lemma 1. For all pairs of states (si, ,52) in case 3 and for all actions e € (i?c(.si, S2) U 
S2)) n Break(si, S2), there exist constraints (pi and (p2 such that Li(si, e, <^i) ^ 
_L and -^2(52, e, (^2) 7^ -L end a distribution /Ui e Sat{(pi) such that either 

1. 3s'i e Si such that Hi{s'i) > and sucCs2,e{si) = 0, or 
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3. 3s'i e ^1,52 £ S2 such that iJLi{s'i) > 0, S2 = sucCs2,e{s'i) and \x\6ti{s'i, s'2) < 

Proof. Let TZ be the maximal refinement relation between Ni and A^2 and let (si , S2) e 

5*1 X 5*2 such that (.si,.S2) is in case 3, i.e. (si,S2) ^ T^and Vi(si) = V2(s2)-Lete e A 
such that e G (i?c(si, S2) U i?/(si, S2)) D Break(si, 52)- 

Since e e Bo(si,S2) U B/(si,S2), there exists e C{Si) and (/?2 € 0(82) 
such that either ^2(52, e, <j£>2) = T and Li(,si, e, 991) = T or L2{s2, e, LP2) =? and 
Li(si, e, ^1) ^ _L. As a consequence, since e G Break(si, S2), we have that 

3fii e S'ai((/9i),V/i2 e Sat{(fi2),l^i <0nk M2- (2) 

Let if be the smallest index such that TZk = TZ- By construction, we know that 
lnd-R,(si, S2) = k < K, i.e. (si,S2) G T^fc and (si,.S2) ^ Tlk+i- Consider the distri- 
bution jjLi given by (2) above. We have that V/j,2 G Sat{ip2),y corresp. (5, /xi /i2. 
Consider the function S such that S2) = 1 if S2 = sucCg^.els'i) and otherwise. 
There are several cases. 

- If there exists s'l € such that i^i {s'l) > and sucCs2,e(s'i) = 0, then the lemma 
is proven. 

- Else, 5 is a correspondence function. Since V/i2 G Sat{(p2), /xi fi2, we know 
that either (1) 1^2 '■ s'2 ^s[eSi A*i(si)^('S'n ■^2) does not satisfy ip2, or (2) there 
exists s'l and S2 such that Hi{s'i) > 0, 6{s[,S2) > and (s'l, S2) ^ 7?.^. 

1. Assume that iJ,2 ■ s'2 J^s'^eSi ^2) does not satisfy <^2- Remark 
that the function /zf from Lemma 1 is equal to fi2 defined above. As a conse- 
quence, III ^ ip2. 

2. Otherwise, assume that there exists s[ and s'2 such that ^1(^1) > 0> '^(^'ii ^2) > 
and {s'l, s'2) ^ T^fe. Since {s'i,s'2) ^ 7?.fc, we have that \ndTz{s'i, s'2) < k. 
As a consequence, there exists s'l G S'l, S2 € "52 such that A*i(s'i) > 0, S2 = 
sucCs2,e(si) and lnd-R,(si, 53) < lnd-R,(si, S2). 

□ 

In other words, the above lemma ensures that a pair {s'l , s'2 ) such that I ndT?, (s'^ , 53 ) = 
can be reached within a bounded number of transitions for all pairs of states (si, S2) in 
case 3. As explained above, this is a prerequisite for the correctness of the counterexample 
construction defined hereafter. 

We now propose the main contribution of the section: a construction to build coun- 
terexamples. Let Ni = {Si, A, Li,AP, Vi, {si}) and N2 = {S2, A, L2, AP, V2, {sg}) 
be deterministic APAs in SVNF such that Ni N2. Let TZ be the maximal refinement 
relation between A^^i and N2.. 

Definition 9. The counterexample P = {S, A, L, AP, V, sq) is computed as follows: 

- S = Si X (S2 U {±}), 

- So = {si, si). 
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Let (pi G C(5'i) such that Li(si,e,ipi) ^ _L and 
let fxi be an arbitrary distribution in Sat{ipi). Define 
-f^((si,S2j,e,/ii ) - 1 . 


Bt,{si,S2) 






Bd[Sl, S2) 


1 -1 




c 


3 


For all /i e Dist{S), let -C/((si, S2), e, /^) = ±. 
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Let (^1 G and (fi2 G C'(5'2) such that 
Li(si,e,(fii) / ± and 1/2(52,6,992) / -L- 

- If e G Break(si, S2), then let /iti be the distribution 
given in Lemma 1 . 

- Else, let ^1 be an arbitrary distribution in Sat{ipi) 
such that V/i2 G Sat{(p2), IJ.1 <$ti y.2. 

In both cases, let I/((si, S2), e, /TT) = T. 


Bf{si,S2) 


}- „,} 



Table 3: Definition of the transition function L in P. 



- y(si, S2) = w G such that Vi(.si) = {v} for all (si, S2) € S, and 

— L is defined as follows. Let (si, S2) G S. 

• If {si, S2) in case 1 or 2 or S2 — -L, then for all a ^ A and ipi G C(S'i) such 
that Li(si, a^ipi) = T, let fii be an arbitrary distribution in Sat{(pi) and let 
^((si, S2), a, fij^) = T with nj^ e Dist{S) such that S2) = Mi(^i) if 
S2 = -L and otherwise. 

• (si, S2) is in case 3 and B{si. 82) 7^ 0. For all a € A \ B{si, S2) and 
(fi € C{Si) such that -Li(si, a,ipi) = T, let jii be an arbitrary distribution in 
Sat{(pi) and let L((si, S2), a, fij-) = T, with fj,^ defined as above. 

In addition, for all e G B{si^S2), let L((si, S2), e, .) be defined as in Ta- 
ble 3. In the table, given constraints (pi G C{Si) and (p2 G 0(82) such that 
Li{si,e,ipi) 7^ J- and L2{s2,e,(p2) ^ -L, and a distribution fii G Sat{(pi), 
the distribution 'jll € Dist{S) is defined as follows: /ii(si, Sj) = IJ-iisi) if 
s'2 = sucCs2,e{si) or sucCs2,e{si) = and s'2 = -L, and otherwise. 



Theorem 4. The counterexample PA P defined above is such that P \= Ni and P y= 
N2. 
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Proof. LetTVi = A, Li, AP, V^i, {sj}) and iVa = (^2, A ^2, ^P, 1^2, {sg}) be 
detemiimstic APAs in SVNF such that A^^i ^ iV2. Let P = (5, A, i, AP, V, sq) be the 
counterexample defined as above. We prove that P \= Ni and P ^ N2. 

P 1= Ni. Consider the relation TZg C S x Si such that (si, S2) T^s s'l iff Si = s'^. We 
prove that TZg is a satisfaction relation. Let t = (si, S2) e 5 and consider {t, si) GlZg. 

- By construction, we have V{si, S2) Q Vi(si). 

- Let a & A and (fi e C(5i such that Li {si, a,ipi) = T. There are several cases. 

• If (si, S2) in case 1 or 2 or S2 = -L, then by construction there exists G 
Dist{S) such that L((si, S2), a, ^5'-) = T. By construction, we have that there 
exists Hi e Sat{(pi) such that i^i (£7^^ ^1. 

• Else, (si, S2) is in caseS and B(si, S2) ^^0. Ifa^ i?(si,S2), the result follows 
as above. Else, either a € Ba{si, S2) U Bi,{si, S2) and the result follows again 
by construction, or a G -Bc(si, S2) U -B/(s 1, S2). In this case, there exists a 
distribution /T^ e Dist{S) such that i((si, 52)5 a, = T. By construction, 
/IT is defined as follows: 



where Hi is either the distribution given by Lemma 1 if a e Break(si, S2) or 
an eirbitrary distribution in Sat{(fii). In both cases, fii G Sat{(pi). Consider the 
function 5 : 5 x S'l ^ [0, 1] such that (5((s;, s'2), s'() = 1 if s'l = s'( and 
otherwise. Using standard techniques, on can verify that ^ is a correspondence 
function and that /Ii (ek, fii. 

- Let a & A and /x G Dist{S) such that L{{si,S2),a, /z) = T. By construction of P, 
there must exists tpi G (^(Si) such that Li(si, a, (^1) ^ _L and is either of the 
form Hi or JIi for some yUi G Sat{ipi). As above, we can prove that in all cases, 
M <E7?,, Ml- 

Finally TZs is a satisfaction relation. Moreover, we have ((sg, Sq)) ^q) ^ '7^s> thus 

P ^ N2. Let T^-s C 5 X 52 be the maximal satisfaction relation between P and N2, 
and assume that TZs is not empty. Let 7?. C 5i x ^2 be the maximal refinement relation 
between Ni and N2 and let K be the smallest index such that TZk = Tl- We prove that 
for all (si, S2) G S'l X S2, if lnd-R,(si, S2) < K, then ((si, S2), S2) ^ IZg. The proof is 
done by induction on fc = Ind7^(si, 52)- Let (si, S2) € 'S'l x ^2. 

- Base case. If \nAn{.si, S2) = 0, then there are several cases. 

• If (si, S2) in case 2, i.e. ^1(51) ^ ^2(52). In this case, we know that V {{si, S2)) 
G Vi(si). Thus, by SVNF of Ni and N2, we have that ^((si, S2)) ^ ^2(82) 
and ((si,S2),S2) ^ Tls. 

• Else, if (.si; S2) in cases 3. a or 3.6, then there exists a ^ A and Hi G Dist{S) 
suchthatiy((.si,S2),a, A*i^) = T and V(/?2 G C(S2), we have iy2(s2, a, ¥'2) = 
_L. As a consequence, ((si, S2), S2) ^ T^s. 




/Xl(si) if s'2 = SUCCs2,e(s'l) 

or sucCs2_e('S'i) = and 53 = -L 
otherwise 
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• Else, if (si, S2) in cases 3. or 3. d, then there exists a G Aand(p2 € C(52) such 
thati2(s2, a, f2) = T and for all /i e Dist(S'), wehave L{{si, S2), a, fj) = _L. 
As a consequence, ((si, S2), S2) ^ ^s- 

• Finally, if (si, S2) incases 3. cor 3./, there exists e S (-Bc(si, S2)U-B/(si, S2))n 
Break(si, 52)- By Lemma 1, there exists constraints ipi and <^2 such that 
^1(51,6, (/Si) ^ _L and 
Z/2(s2, e, (/32) 7^ -L and a distribution /ii e Sat{(pi) such that either 

(I) 3si e S*! such that fJ,i{s[) > and sucCs2,e(si) = 0, or 
(H) M? : (s'2 ^ E{.; (s'l)) ^ 5at(^2), 

(III) 3s'i e Sij-Sa e 5*2 such that fii{s'i) > 0, S2 = sucCs2,e(s'i) and 

lnd7?,(si,S2) < lndK(si,S2). 
By construction, we have that L((si, S2), e, jtli) = T for fii given above. Since 
Ind7^(si, S2) = 0, case (III) above is not possible. From cases (I) and (II), we 
can deduce that for all /i2 € Sat{(p2), we have /IJ /i2- Moreover, by 

determinism of N2, f2 is the only constraint such that .^2(52, e, </52) 5^ -L. As a 
consequence, ((si,S2),S2) ^ 'T^s- 

- Inductive step. Let < k < K and assume that for all fc' < A; and for all 

{■s[,s2) € §1 X S2, if Ind7j(,si, ,S2) = fc', then ((si,S2),S2) ^ T^s- Assume that 
\ndTz{si, S2) = k. There are two cases. 

• If (.si, ,S2) in cases 2, 3. a, 3.6, 'i.d or '3.d, the same reasoning applies as for the 
base case. We thus deduce that ((si, S2), S2) ^ T^s- 

• Otherwise, if (si,S2) in cases 3.c or 3./, then, as above, there exists e e 

{Bc{si, S2) U Bf{si, S2)) n Break(,si, S2). By Lemma 1, there exists constraints 
ipi and <^2 such that Li(si, e, (pi) 7^ _L and L2(s2, e, (^2) 7^ -L and a distribu- 
tion /xi e Sat{(fii) such that either 

(I) 3s'i e 5i such that /Ui(s'i) > and sucCs2,e(si) = 0. or 

(III) 3s[ e 5'i,S2 G 52 such that Ati(s'i) > 0,S2 = sucCs2,e(s'i) and 
Ind7^(s;,s'2) < lnd7^(si,S2). 

By construction, we have that i/((si, S2), e, fli) = T for ^1 given above. As 
above, if cases (I) or (II) apply, then we can deduce that ((.si, S2), S2) 4- T^s- 
If case (III) applies, then there exists {s'l, s'2) € S such that JIi{s'i, s'2) > 0, 
s'2 = sucCs2,e(si) and \r\(iTz{s[, s'2) < lndK(si,S2). Since S2 = sucCs2^e(s'i), 
then, by determinism of N2, all correspondence functions 5 will be such that 
(5((.s'j, .S2), S2) = 1. However, we have that I nd-R,(s'i, S2) < A;, thus by induction 
{{s'l, s'2}, s'2) ^ T^-g. As a consequence, we have that for all /U2 € Sat{ip2),we 
have /xT ^-jZs M2- We can thus deduce that ((si, S2), S2) ^ T^s- 

Finally, we know that I nd7j(so,So) < fc. As a consequence, we have ((sqj Sq), Sq) ^ 
and thus P ^ N2. 



□ 
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Appendix: Proofs 
Proof of Proposition 1 

For all APAs Ni and N2 in SVNF, it holds that dt {Ni , ^^2) < d{Ni , 7V2) ■ For a distri- 
bution /Ui and a constraint <^2, we denote by 

RD(/ii,(p2) := {S : Hi (s^ M2 I S Sat{ip2)} 

the set of all simulations between /xi and distributions satisfying (p2. 

Proof. If d{Ni,N2) = 1, we have nothing to prove. Otherwise, write Ni = (Si, A, Li, 
AP,Vi,S'o) for i = 1,2, and let Pi = {S{, A, L{, AP,V{, S^) G |iVi] and ry > 0; 
we need to expose P2 G IN2} for which (i(Pi, P2) < d{Ni,N2) + rj. Note that by 
the triangle inequality, d{Pi,N2) < d{Pi,Ni) + d{Ni,N2) < d{Ni,N2). Define 
P2 = (5*2, A, L'^,AP, V2,S^), with L'^ given as follows: 

For all s[ G S[, a G A, /xi e Dist{S[) for which L[{s'i, a,fj,i) = T and for all 
S2 € 5*2, £ < 1 with e := d(si,S2) < 1: We must have (p2 € Dist{S2) such that 
L2{s2,a,ip2) ^ -L and 

so there must exist a redistribution 5 e RD(/ii , (^2) for which 

^ A*i(ii)<5(ii,i2)(i(ti,t2) < A-^e + A-^r/. 
(ti,t2)es;xS2 

Welet/X2(s) = X^s'^eSi Mi('Si)<5(sii ■«) and set L2(s2, a, ,^2) = TinP2. 

Similarly, for all .S2 G ^2, a G A, Lp2 0(82) for which ^2(52, a, 1^52) = T and 
for all .s'^ e S[ with e := d(si,S2) < 1: We must have /xi G Dist{S'i) for which 
Li(si, a,/zi) = T and 

inf V ^il{t\)S{t[,t2)d{t[,t2) < X-^s , 

so there is 5 G RD(/ii, 1^2) with 

^ Mi(t'i)^(t'i,t2)rf(i'i,t2) < A-i£ + A-^r/. 
(ti,t2)es;xS2 

Let again /U2(s) = Zls'^gSj ij,i{s[)S{s[, s), and set ^2(52, a, H2) = T in P2. 

It is easy to see that P2 G |A^2l : by construction of P2 , the identity relation {(52,^2) | 
S2 G 5*2} provides a refinement P2 < N2. To show that (i(Pi, P2) < d{Ni,N2) + r?, 
we define a function d' : S[ x S2 [0, 1] by d'{s[,S2) — d{s[,S2) + rj and show that 



Refinement and Difference for Probabilistic Automata 



23 



d! is a pre-fixpoint to (1). Indeed, for s'-^ and S2 compatible, we have 
d'(s'i,S2) = d(si,S2) + r? 



max min XDp^^N^{iii,(p2,d) + v 

max 



=T jLii:L'j(sj,a,(ii)=T 



= max < 



max min ADp^ (/xi, /X2, c?) + 

o,/:ii:Lj(sj,a,/ii)=T /H2;i2(*2,<i,/i2)=T 

max min ADp^^p^ (/ii, /i2, c?) + ?7 , 

a,/:j2:i2(*2,a,;i2)=T /:ti:Lj(sj,a,jUi)=T 



due to the construction of P2 and the fact that the sup^^g^^j^^^j is trivial in the formula 

forDp^,jV2(Mi> V2,rf), 



> max < 



max min XDp^^p^ipi, fi2,d') 

a,^tl■.L[{s[,a,^ll)=T /H2:i2(*2,a,M2)=T 

,max , ^ AL»Pi,P2(/xi,/X2,rf') > 

a,/H2:i'2(s2,o,(i*2)=T /ni:I,j(Sj,a,jUi)=T 



where the last inequality is a consequence of 

AI)p„p, {111,112, d') = Xj2 Mt'iWi,t2){d{t\,t2) + rj) 

= A ^ Hi{t[)6{t[,t2)d{t[,t2) + Xv- 

t'l ,t2 



□ 



Proof of Theorem 1 



For all deterministic APAs A^i and N2 in SVNF such that Ni y< N2, we have {Nij \ 

IN2I c liVi \* 7V2I. 

Proof. Let iVi = {Si, A, Li,AP, Vi, {sj}) and N2 = {S2, A, L2, AP, V2,{sl}) be de- 
terministic APAs in single valuation normal form such that N^:^ N2. Let TZ be the maxi- 
mal weak refinement relation between A^i and Af2- LetP = {Sp, A,Lp, AP, Vp, Sq) be 
a PA such that P iVi and P ^ A^a- We prove that P ^ Ni \* N2. Let UiC Spx Si 
be the relation witnessing P \= Ni and let TZ2 be the maximal satisfaction relation in 
Sp X S2. By construction, {sq, S2) ^1^2- 

If Vi{sl) ^ V2{sl), then by construction Ni \* N2 = Ni and thus P \^ Ni\* N2. 
Else, we have (sq, Sq) in case 3, thus A^i \* N2 = {S, A, L, AP, V, Sq) is defined as 
in Section 4.1. By construction, we also have {sq, Sq) in case 3, thus there must exist 
/ e B{s^, Sq). Remark that by construction, we must have B{s 
will prove that P ^ A^i \* iV2. 

Define the following relation TZ^ C Sp x S: 
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{{pTZi si) and (s2 = -L) and (e = e) 
or {pIZi si) and {p, S2) in case 1 or 2 and and (e = e) 
or {pTZi si) and {p, S2) in case 3 and (e e -B(p, S2)) 

We now prove that TZ^ is a satisfaction relation. Let {p, (si, S2, e)) € 7?.\ 
If S2 = ± or e = £, then since p7?.i si, 7?.^ satisfies the axioms of a satisfaction relation 
by construction. 

Else we have S2 € 52 and e ^ s, thus, by definition of we know that {p, S2) is in 
case 3. 

- By construction, we have Vp{p) G Vi(si) = V((si, S2, e)). 

- Let a G A and yUp e Dist{Sp) such that Lp{p,a, fxp) = T. There are several 
cases. 

• Ifaj^e, then sincepT^i Si, there exists ipi € C(<S'i) such that Li(si, a, i^i) ^ 
_L and there exists € Sat{(pi) such that /ip (s^^x /ii. By construction, we 
have L{{si,s2,e),a, (pj^) ^ _L and there obviously exists fi e Sat{ifj^) such 
that /xp fi. 

• If a = e e Ba{p,s2), then, as above, there exists 93 G C(5) such that 
-f'((si, S2, e), a, (^) ^ _L and there exists n G Sat{(p) such that ^p /x. 
Remark that Ba(s 1,52) C Ba{p,S2) Q Ba{si,s2) U Bfc(si,s2). 

• Else, we necessarily have a = e G i?c(p, S2) U -B/(p, S2). Remark that, by 
construction, Bc{p,S2) C Bc{si,S2) and Bf{p,S2) C Bf{si,S2)- Since 
p7?.i Si, there exists G C(S'i) such that Li(si, e, (pi) ^ ± and there exists 
/zi G Sat{(pi) and a correspondence function (5i : S'p — ^ (^i — ^ [0, 1]) such 
that fip (S^^ fj,i. 

Moreover, by construction of Ni\*N2, we know that the constraint Lp f2 such that 
H G Sat{ipf2) iff- (1) for all (s'l, fi) G S, we have ^(s^, 4, c) > ^ sJ, = 
_L if sucCs2,e(si) = and s'2 = sucCs2,e(s'i) otherwise, and c G B{s[,s'2) U 
{e}, (2) the distribution fii : s[ ^ EceAu{e},s^eS2U{_L} /"(^'i- 4; c) satisfies 

and (3) either (b) the distribution /i2 : s'2 ^-^ J2ceAu{£} s' eSi /-'(■^'i: -^[i' 
does not satisfy (f2, or (c) there exists s[ G Si, s'2 G 52 and c ^ e such that 
^(s'l, s'a, c) > is such that L{{si,S2, e), e, (pf2) = T. 

We now prove that there exists fi G Sat{ipi2) such that /ip (g^jx /i. Consider 
the function S"^ : Sp {S ^ [0, 1]) defined as follows: Letp' G Sp such that 
Hp{p') > and let s'^ = sucCsi.eip'), which exists by TZi. 

* If succ32,e(p') =0, then(5\(p')(.s-i,-L,£) = 1. 

* Else, let s'2 = sucCs2,e(p')- Then, 

• if {p', s'2) G 7e2,'then ,5\(p')(s'i, 4, e) = 1. 

• Else, (p',.S2) is in case 3 and i?(p',S2) 7^ 0- in this case, let c G 

and define (5\(p', (si,S2,c)) = 1. For all other c' G ^(p',^), 
define (5\(p',K,, s'2, c))=0. 
Remark that for all p' G Sp such that iJ.p{p') > 0, there exists a unique s' G 5' 
such that 6\{p'){s') = 1. Thus S\ is a correspondence function. 
We now prove that /x = iipS^ G 5af(i^f2). 
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1. Let (si,S2,c) e S such that ^{s'i,s'2,c) > 0. By construction, there 
exists p' e Sp such that i-J.p{p') > and 6'\{p'){s[, S2, c) > 0. Moreover, 
c G B{s[, s'2)U{s}, md s'2 = -Lif sucCs2^e(s'i) = 0ands2 = sucCs2^e(s'i) 
otherwise. 

2. Consider the distribution : s[ ^ EceAu{e},46S2U{_L} /"(s'l, 4> c). By 
determinism (See Lemma 28 in [8]), we have that 5i{p')(s'i) = 1 ■^=^ 
s'l = {succ)sT^,e{p')- As a consequence, we have that ji'i = /zi G Sat{(fi). 

3. Assume that for all p' G Sp such that Hp{p') > 0, we have sucCs2,e{p') 
(the other case being trivial). Consider the distribution /i2 : 
EceAu{e},si6Si M(si,S2>c) and let ^2 : 5'p -j> {S2 [0, 1]) be such 
that S2{p'){s'2) = 1 s'2 = sucCs2,e{p')- By construction, 62 is a cor- 
respondence function and /i2 = fJ-pS2- Since e G -Bc(p, S2) ^ Bf{p, S2), we 
have that /zp ^7^2 ^2- If M2 ^ Sat{ip2), then we have /it € Sat{ipf2)- Else, 
there must exist e Spandsj G (S'2 such that /ip(p') > 0, (52(p')(*2) > 
and (p', S2) 1^ 712- As a consequence, (p', .sJ,) is in case 3 and there ex- 
ists c ^ e such that 5^{p'){s'i,s'2,c) > 0, thus iJ,{s[,s'2,c) > 0. As a 
consequence, /x G Sat{(p 12). 

We thus conclude that there exists fi E Sat{(ff2) such that /ip (It^x /i. 
Finally, in all cases, there exists (f G C{S) such that L((si, S2, e), a, ly?) ^ _L and 
there exists G Sat{ip) such that /Up (s^^x /U. 
- Let 0, G A and (f G C(5') such that L{{si, S2, e), a, = T. As above, there are 
several cases. 

• If a ^ e, then, by construction of Ni \* N2, there must exists ifi G C{Si) such 
that Li{si,a, (pi) = T. The rest of the proof is then as above. 

• Ifa = eG Bo (p, S2), then there exists /zp G £'isf(S'p) suchthatLp(p,e,/ip) = 
T. The rest of the proof is then as above. Recall that Ba(si, S2) C Ba{p,s2) Q 

E„(si,.S2)USb(si,.S2). 

• Else, we necessarily have a = e G Bc{p,S2) U Bf{p,s2)- Recall that, by 
construction, Bc{p, §2) C i?c(si, S2) and S2) C i?j(si, S2). Thus, there 
exists /xp G Dist{Sp) and (^2 G C(S'2) such that L2{s2,e,(p2) 7^ -L and 
V/X2 G Sat{(p2),np <$n2 M2- Since e G i?c(si,S2) U Bf{si,S2), there also 
exist <pi G C(5i) such that Li{si,e, ipi) 7^ -L. By determinism, ipi and (/72 are 
unique. The rest of the proof follows as above. 

Thus, in all cases, there exists /ip G Dist{Sp) such that Lp{p,a, fj.p) = T and 
there exists /x G Sat{ip) such that /ip <£j^\ /x. 

Finally, 7?.^ is a satisfaction relation. Moreover, we have Sq TZi Sq, (sq , Sq) in case 
3 and / G B{sq, Sq) by construction, thus Sq' ^H^oj /) ^ '^o- 
We thus conclude that P |= iVi \* JV2. 

□ 

Proof of Theorem 2 

For all deterministic APAs Ni and N2 in SVNF such that N2, we have that 

1. for all ii- G N, |7Vi N2\ C |7Vil \ |iV2], and 
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2. for all PA P e |iVi] \ IN2], there exists K eN such that P e {Ni \^ iVa]. 

Proof. For the first claim, consider the relation Tl C {Si x {S2 U {-L}) x {AU {e}) x 
{!,..., K}) X (^i X (52 U {_L}) X (A U {s}) x {1, . . . , K + 1}) such that TZ = 

{(( 

Sg, Sq, e, K), (sj, Sq, e, -R'+l)) | e G -B(so, Sg)}U7^id, where T^id denotes the identity 
relation. One can verify that, by construction, 7?. is a refinement relation witnessing 
Ni 7V2 ^ Ni TVs. 

LetTVi = {Si,A,Li,AP,Vi,{sl}) and N2 = (^2, A ^^2, ^P, X^2, {sg}) be de- 
terministic APAs in single valuation normal form such that A*"i ^ A^2- Let TZ be the 
maximal weak refinement relation between Ni and A^2- 

1. We first prove that for all K [iVi \^ 7V2I Q {Ni} \ [iVa]. 
If Viisl) ^ V2(sl), then for all G N, we have Ni \^ N2 = Ni and the result holds. 
Otherwise, assume that (sj, Sq) is in case 3 and let K G N. We have A^i \^ A^2 = 
(5, A, L, AP, V, S^) defined as in Section 4.2. Let P = (Sp, A, Lp, AP, Vp, s^) be a 
PA such that P |= Ni \^ N2. Let 7^^ C S'p x 5* be the associated satisfaction relation 
and let / e B{sl, sg) be such that 7^\sJ, sg, /, K). We show that P ^ A''i and 

P ^iV2. 

We start by proving that P \= Ni. Consider the relation TZi C Sp x Si such that 
pTZiSi <^ 3s2 e {S2U {±}),3e e {AU {e}),3n < K s.t. p7l^{si,s2,e,n). 
We prove that TZi is a satisfaction relation. Let p, si,S2,e,n such that pTZi si and 
p7^\(sl,S2,e,n). 

- By construction, we have Vp{p) G V{{si,S2, e, n)) = Vi{si). 

- Let a € A and /ip G Dist{Sp) be such that Lp{p, a, fip) = T. By 7?.\ there exists 

e C{S) such that L((si, S2, e, n),a,(f) ^ _L and there exists G Sat{(p) such 
that /Up /It. 

If S2 = _L or e = t or a 7^ e, then by construction of Ni \^ N2, there exists (pi G 
C(5'i) such that (f = (p^ and Li (si, a,(pi) ^ _L. As a consequence, the distribution 
/i 4,1: s'l /Lt(s'j, _L, e, 1) is in Sat{(fi) and it follows that /zp (=7^^ ^ 4,1. 
Otherwise, assume that S2 G 6*2, e G A and a = e. There are several cases. 

• If e G PnC-si: S2) U Ph(si, S2), then by construction of A^i \^ N2, there exists 
(fii G C(S'i) such that Li(si, e, ipi) ^ _L and ip = pj-. As above, we thus have 

• Else, if e G Bf.{si, S2), then there exists ipi G (^(^i) and (/52 & C{S2) such that 
I/i(si, e, (fii) =? and ^2(52, e, (/?2) = T. Moreover, is of the form (/sfj, and 
n' G S'at(<^j^) implies that the distribution such that 
Mi : EceAu{e},s^eS2U{±},fe'>iA*(si,S2,c,fc') satisfies </?i. Thus, the 
distribution /xi : s[ ^ EeeAu{e},4eS2U{±},fe'>i mK, 4, c, fc') satisfies (pi. 
Let 5i : Sp ^ (S'l [0, 1]) be such that 6i{p'){s'i) = 1 if /ip(p') > and 
s'l = sucCs-^,e{p') and otherwise. By construction, 5i is a correspondence 
function and we have iipSi = /Ui. 

Thus there exists ^i G Sat{(pi) such that /Up g-^j /xi. 

• Finally, if e G Pc(si, ^2) U Bf{si, S2), then there exists 931 G C(Si) such that 
L(si, e, </5i) ^ _L, and either (p = (p^ 01 (p = as in the case above. In both 
cases, as proven before, there exists /xi e Sat{ipi) such that /xp €7^^ /ii. 



Refinement and Difference for Probabilistic Automata 27 

- Let a E A and (pi e C{Si) such that Li{si,a, (pi) = T. 

If S2 = -L or e = e or a ^ e, then by construction of A''i \^ N2, the constraint (pj^ is 
such that L{{si, S2, e, n), a, 955'-) = T. As a consequence, there exists a distribution 
Hp e Dist{Sp) such that Lp{p, a, ^p) — T and there exists /x G Sat{ipi ) such 
that /xp /i. Moreover, by construction of (fj-, the distribution /i 4,1: s[ i-)- 

/x(s']^, _L, £, 1) is in S'at((pi) and it follows that /ip (etj^ 

Otherwise, assume that S2 G ^2, e E A and a = e. Since Li(si,a, = T, 
(si, S2) can only be in cases 3. a, 3.c or 3./. As a consequence, e e Ba{si,S2) U 
-Bc('Si;S2) U i?/(si,S2). By construction, in all of these cases, we have 
L{{si,S2,e,n),a,(pi) = T. Thus, there exists a distribution /xp e Dist{Sp) 
such that Lp{p, a, hp) = T and there exists /U e Sat{ipi) such that /Up /x. 
As above, it follows that /xp (et^j /x 4i. 

Finally, TZi is a satisfaction relation. Moreover, by hypothesis, we have 

sP n\{sl si /, K), thus sP Til si and P ^ iVi. 

We now prove that P |^ A'2. Assume the contrary and let TZ2 C Sp x S2 be 
the smallest satisfaction relation witnessing P \= N2 (i.e. containing only reachable 
states). We prove the foUowdng by induction on the value of n, for 1 < n < K: 
Vp e Sp, S2 G S2, if there exists si e Si and e € A such thatpT^-^si, S2, e, n), then 

(P,S2)^7^2. 

- Base Case (n = 1). Let p, si, S2, e such that p7^\si, S2, e, 1). If e € -Ba(si) ^2) U 
S6(si, S2)Ui3(i(si, S2), then by construction there is an e transition in either P or A^2 
that cannot be matched by the other. Thus (p, .S2) ^ The same is verified if e € 
Pe(si, S2) and there is no distribution ^p e Dist{Sp) such that Lp{p, e, jip) = T. 
Else, e e Pe(si, S2) U Pc(sij S2) U i?/(si, S2) and there exists /xp e Dist{Sp) 
such that Lp{p, e, /xp) = T. Let ipi G C{Si) and (/32 € 0(82) be the corresponding 
constraints in iVi and A^2- Consider the corresponding constraint (pf^^ G C(S'). By 
7?.\ there exists /x G Sat(Lp^2^) such that /;xp (s^jx /x. By construction of (Pi2^, we 
know that either (3.a) there exists {s[, _L, s, 1) such that /j,(s'^ , _L, e, 1) > or (3.b) 
the distribution /i2 : sJ, SceAu{e} s' eSi k'>i f^i^'i^ ^2' '^'^^^ ^'^^ satisfy ip2- 
If there exists (s'^, _L, £, 1) such that iJ,{s[,-L, e, 1) > 0, then there exists p' G Sp 
such that Hp{p') > and sucCs2,e(p') = 0- Thus there cannot exists /X2 G Sat{ip2) 
such that /ip (£7^2 /ij. Otherwise, by determinism of N2, we know that the only 
possible correspondence function for fip and TZ2 is 62 ■ Sp ^ {S2 [0, 1]) such 
that 52{p'){s'2) = 1 if S2 = sucCs2^e(p') ™d otherwise. By construction, we have 

= M2 and thus there is no distribution G Sat{ip2) such that /xp (s^j /X2. 
Consequently, (p, S2) ^ TZ2. 

- Induction. Let 1 < n < K and assume that for all k < n, for all p' G Sp, S2 G ^2, 
whenever there exists s[ G S'l and e E A such that p' Tl^{s[, s'2, e, k), we have 
(p':'S2) ^ 7^2- Let p, si, S2, e such that p7?.\si, S2, e, n). If e G Pa(si,S2)U 
Bb{si,S2) U i3d(si,S2), then by construction there is an e transition in either 
P or iV2 that cannot be matched by the other. Thus (p, S2) ^ T^2- The same is 
verified if e G i?e(si, S2) and there is no distribution fip G Dist{Sp) such that 
Lp{p, e, /Xp) = T. Else, e G -Be(si5 ^2) U i?c(si, S2) U P/(si, S2) and there exists 
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lip G Dist{Sp) such that Lp{p, e, jip) — T. Let (^i S C{Si) and S C{S2) be 
the corresponding constraints in A^i and N2. 

Consider the corresponding constraint t/?^'" G C{S). By 7?.\ there exists /i € 
^^^((^fg") such that /zp (et^x /i. By construction of ^Pi2 \ we know that either 
(3. a) there exists (s'l, -L, c, 1) such that ^(s'^, _L, c, 1) > or (3.b) the distribution 
M2 : S2 EceAu{e},yiGSi,fc'>i S2, c, A;') does not satisfy (^2, or (3.c) there 
exists s'l e 5i, S2 G 5*2, c 7^ e and k < n such that /i(.si , Sj, c, /c) > 0. If case 
(3. a) or (3.b) holds, then as in the base case, there is no distribution /Xj S Sat{ip2) 
such that /Up €7^3 /Li2. Otherwise, if (3.c) holds, then there exists p' e Sp such that 
jJbpij)') > and p' TZ^{s[, ^'2, c, k). By induction, we thus know that {p' , s'2) ^ TZ2 
and by construction and determinism of N2, we have that sucCs2,e{p') = {52}- 
Thus there is no distribution /U2 € Sat{ip2) such that /ip €7^2 M2- Consequently, 

(P,S2) ^:/^2. 

By hypothesis, we have Sq TZ^{sl, Sq, f, K). As a consequence, we have that 
(s^, sg) ^ Tea, implying that P ^ iVa. 

2. We now prove that for all PA P e \ IN2], there exists K G N such that 

P€lNi\^N2l 

If Vi{sl,) + V^2(so)' then for all X G N, we have iVi \^ N2 = Ni and the result 
holds. 

Otherwise, assume that (sj, Sq) is in case 3. Let P = {Sp, A, Lp, AP, Vp, Sq) be a PA 
such that P \= Ni and P ^ A'^2- Let TZi be the satisfaction relation witnessing P |= A''i 
and TZ2 be the maximal satisfaction relation between P and N2. Assume that TZ2 is 
computed as described in Section 5. Let IndT^^ be the associated index function and 
let K be the minimal index such that 7^2 k = 7^2- We show that P \= Ni\^ N2. Let 
TVi \^ N2 = {S, A, L, AP, V, Sq) be defined as in Section 4.2. 

Let TZ^ C Sp X S2 be the relation such that 



p'Ry{si,S2,e, k) 



or 



or 



[pTZi si) and (s2 = -L) and (e = e) and {k = 1) 
(pT?-! si) and (p, S2) in case 1 or 2 and (e = e) 
and (fc = 1) 

[pTZi si) and (p, S2) in case 3 and (e e Break(p, S2)) 
and (fc = lnd-R,2 (p, S2) + 1) 



Remark that whenever {p, S2) is in case 3, we know that IndTjj [p, S2) < K, thus 
lnd7^,(p,S2) + l < K. 

We prove that 7?.^ is a satisfaction relation. Let p7?.\si, S2, e, fc). 

If S2 = -L or e = £, then since p7?.i si, 7?.^ satisfies the axioms of a satisfaction relation 

by construction. 

Else we have .S2 G S'2 and e 7^ e, thus, by definition of 7e\ we know that [p, S2) is 
in case 3. The rest of the proof is almost identical to the proof of Theorem 1. In the 
following, we report to this proof and only highlight the differences. 
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- By construction, we have Vp{p) S Vi(si) = V{{si, S2, e, k)). 

- Let a G A and fxp G Dist{Sp) such that Lp{p,a, fxp) = T. There are several 
cases. 

• li a e, or a = e G Ba{p, S2), the proof is identical to the proof of Theorem 1. 

• Else, we necessarily have a = e E Bc{p,S2) U Bf{p,S2). Remark that, by 
construction, Bc{p,S2) C Bc{si,S2) and Bf{p,S2) C Bf {31,82). Since 
pTZi si, there exists cpi e such that Li{si,e, (fi) ^ _L and there exists 
fii e Sat{(pi) and a correspondence fimction di : Sp ^ {Si [0, 1]) such 
that fip <E^^ /ii. 

Moreover, by construction of A^i \^ N2, we know that the constraint ^fj*^ is 
such that L{{si,S2, e, k), e, </'f2'^) = T. 

We now prove that there exists ^ G Sat{(pf2'^) such that /ip (e^^v /i. Consider 
the function 5 : Sp ^ {S ^ [0, 1]) defined as follows; Letp' G Sp such that 
Hp{p') > and let s'^ = suiCCsi,e{p'), which exists by TZi. 

* If sucC52,e(p') =0, then5(p')(si,_L,£,l) = 1. 

* Else, let s'2 = sucCs2,e{p')- Then, 

• if {p', s'2) e Tea.'then s'2, £, 1) = 1. 

• Else, {p' , s'2) is in case 3 and Break(p', s'2) 7^ 0. In this case, let c G 

Break(p', S2) and define (s^, S2) c, lndK2(p', S2) + 1)) = 1- For 
all other c' G A and 1 < fc' < /C, define 5{p', (si, 4, c', k')) = 0. 
Remark that for all p' G Sp such that ^J,p{p') > 0, there exists a unique s' G S" 
such that 6{p'){s') = 1. Thus J is a correspondence function. 
We now prove that fi = fxpS G Sat{ipi2^). 

1. Let {s'l. s'2. c. k') G 5 such that fi{s'i, s'2, c, k') > 0. By construction, there 
exists p' G Sp such that n,p{p') > and 5{p'){s'i,s'2, c, k') > 0. Moreover, 
c G B(si, S2) U {s}, s'2 = -L if sucCs2^e(s'i) = and s'2 = sucCg^^eCs'i) 
otherwise. 

2. Consider distribution /x'l : ^ EceAu{£},4eS2U{-L},fe'>i M(si, S2, c. A;'). 
By determinism (See Lemma 28 in [8]), we have that {p') {s'l) = 1 <s=^ 
s'l = {succ)s-i^,e{p'). As a consequence, we have that /xi = fjbdi = /Ui G 
<S'af(^i). 

3. Depending on k, there are 2 cases. 

• If /c > 1, assume that for all p' G S'p such that np{p') > 0, we have 
sucCs2,e{p') 7^ (the other case being trivial). Since c G {Bc{p, S2) U 
Bf{p, S2)) n Break(p, S2) by 7?.\ we can apply Leinma L As a conse- 
quence, either (2) the distribution /if such that 

l4 ■ (s2 ^ Ep'GP I 4=succe2,c(p') M^^(P')) does not satisfy (fi2, or 
(3) there exists p' G S'p and S2 ^ ^'2 such that np{p') > 0, S2 = 
succs2,e(p') and Ind-^Jp', 4) < Ind-T^Jp, S2). 

In the first case (2), consider the distribution 1^2 defined as follows: 

H2:s'2^ XI M(s'i,S2,c,fc'). 

ceAu{E},sieSi,fc'>i 

We have the following: for all S2 G S2, 
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M2(S2) = X] M(Sl>S2)C,fc') 

ceAu{E},s[eSi,k'>i 
ceAu{E},s[eSi,k'>ip'eSp 

= E ^'p(p'^ E sip'){{s[,s'„c,k')) 

p'eSp cGAu{e},s[eSi,k'>l 
= E Mp(p')'^(p')((sUCCsi,e(p'):47C, 

p'eSp I S^=SUCCs2,e(p') 

lndK,(p',s'2))) 
for c € Break(p', S2) fixed as above 

E t'pip') 

p'&Sp I S2=SUCCa2,e(p') 

= M?(4) 

As a consequence, /X2 ^ Sat{(p2) and /x € Sat{ip^2'). 

In the second case (3), we have (5(p')((si, S2) c, fc')) > for s'l = 
sucCsj^e(p'), c G Break(p', s'2) fixed above, and k' = Ind^^ (p', S2) + 
1 < Ind7j2(p, S2) + 1 = A;. As a consequence, we thus have 
^(.s'^, ,S2, c, fc') > for fc' < A; and c 7^ e, thus e Sat{ipi2^). 
* On the other hand, if k = 1, then Ind^^ (Pj -''2) = and either (1) there 
exists p' € Sp such that iJ.p{p') > and sucCs2,e{p') = 9, or (2) the 
distribution the distribution /if such that 

Ml : (4 ^ Ep'ep I 4=succ.2,e(p') Mp(p')) ^ </'2. In both cases, as 
above, we can prove that 11 € Sat{ipl2^'''. 
In both cases, we have /x G Sat{ip^2^^. 
We thus conclude that there exists n G Sat{ip-^2 ) such that /xp (^^\ ^. 
- Let a G A and (f G C(S') such that £((.si. .S2, e), a, (p) = T. As in the proof of 
Theorem 1, there are several cases that all boil down to the same arguments as above. 

Finally, 7^^ is a satisfaction relation. 

Letc G Break7^2(s^, Sq) and consider the relation 7?.^ = 7?.^ U{(s^, (sj, Sq,c,K))}. 
Due to the fact that K > Indv^^ {sq, Sq), one can verify that the pair {sq, (sJ, Sq, c, K)) 
also satisfies the axioms of a satisfaction relation. The proof is identical to the one 
presented above. As a consequence, 7^^ is also a satisfaction relation. Moreover, we now 
have that (s^, (sj, s§, c, K)) G 7^\', with (sj, sg, c, K) G 5o, thus P |= A^i \^ N2. 

□ 

Proof of Proposition 2 

Let iVi and N2 be two deterministic APAs in SVNF such that A^i ;^ A^2, and let K gN. 
Then dt([iVil \ [iV2], {N, \^ N2I) < A^(l - XyK 
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Proof. By Lemma 2, we know that d{Ni_ \^+^ N2,Ni \^ N2) < for each L, hence 
also dtilNi \^+^ N2I {Ni \^ N2I) < for each L by Proposition 1. Applying the 
triangle inequaUty for dt, we see that 



Proof of Theorem 3 

Let A^i and N2 be two deterministic APAs in SVNF such that Ni :/< N2. The following 



1. the sequences [Ni \^ N2)k and (|A^i \^ A^2l)if both converge, 

2. limx^oo dtilNil \ IN2I {Ni \^ N2\ = 0, and 

3. limif^oo d{Ni \* N2, Ni \^ N2) = 0, so that 

Proof. LetiVi = {Si, A, Li, AP,Vi, {s^}) md N2 = (82, A, L2, AP,V2, {si}) be 
two deterministic APAs in SVNF such that Ni N2. 

1. The proof of the convergence of both sequences (iVi \^ N2)k and (lA^i N2\)k 
is done as follows. We show in Lemma 2 that the sequence (A^i \^ N2)k is bi-Cauchy 
(i.e. both forward-Cauchy and backwards-Cauchy) in the sense of [7]). 

Lemma 2. Let Ni = (S*!, A, Li, AP, Vi, {sj}) and N2 = {S2,A,L2,AP,V2,{sl}) 
be two deterministic APAs in SVNF such that Ni N2. Let 1 < Ki < K2 be integers. 
The distance between Ni N2 and Ni \-^^ N2 is bounded as follows. ■ 

d{Ni N2, Ni N2) < . 

Proof Let A^i N2 = = {S\ A, L\AP, V\ Tq'). The proof is in several steps. 

- We first remark that for all (si, S2, e) e S'l x {S2 U _L) x (^U e) and for all k < Ki, 
the distance between State (si, S2, e, fc)^ G and (si, S2, e, k)"^ G S"^ is 0. Indeed, 
if k is the same in both states, then they are identical by construction. 

- We now prove by induction on 1 < /ci < Ki and fci < A;2 < K2 that 
d((si,S2,e, fc2)^ (si,S2,e, A;i)i) < A^^^. 

• Basecase:fci = 1. By construction, = (si, S2, e, fci)^ andt2 = (si, S2, e, ^2)^ 
have the same outgoing transitions. The only distinction is in the constraints 
(pf2^ and (fif-^''^ when e € -Bc(si, S2) U i3e(si, S2) U i?/(si, S2). As a conse- 
quence, the states ti and t2 are compatible, thus 



00 



i=0 



00 .j^ 
i=0 



□ 



holds: 



d{t2,ti) = max < 




max I min 
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Moreover, we know by construction that D^2^]^i {ip', (p,d) < 1 for all (p' and 
(f. As a consequence, d{t2,ti) < A = A'^^ 

• Induction. Let = (si, S2, e, fci)^ and ^2 = (si, S2, e, A;2)'^, with 1 < fci < 
k2. Again, if e ^ Bc(si,S2) U Be(si,S2) U Bf{si,S2), then ti and t2 are 
identical by construction and the result holds. Otherwise, the ^air of constraints 
for which the distance is maximal will be constraints (pi2 ^ € C{S^) and 
iff^''^ e C(52). Assume that rf((si,S2,e,fc^)2,(si,S2,e,fc'i)i) < A*^^ for all 
k[ < ki and k[ < k'2 < K2. By definition, we have 



ri / B,k2 B,ki j\ 

-t^JV2,JVH¥'l2 '¥'12 '"j = 



sup 



inf 

5eRD(;^2,vf2''') 



E f^2{t'2)S{t'2A)d{t'2,A) 



Consider the function S : S'^ x [0, 1] such that 



'1 ifs[ 



5{{s\,s'2J,k'2),{s'l,4,f',k[))={ 



s'2' A /' = / 
Afc'i = k2Ak'2 < ki 
1 ifs; = s'/A4 = s'2'Af = / 

Afci = All - 1 A /ci < ^2 
otherwise 



Let 1^2 S Sat{ifii2'^'^). One can verify that 6 G RD(/i2, 'y^f2'^^) as follows: 



1. Let t'2 = {s[,s'2, /, k'2) be such that /U2(i2) > 0- definition, we always 



haveX;*^^^! (5(i'2,i'i) = 1. 



2. (5 preserves all the conditions for satisfying (^^2'^'^ . In particular, all states 

t'2 = (s'l, S2, /, ^2)^ such that k'2 < k2 are redistributed to states 
{s'i,s'2, /, fci)^ with A:^ < fci. As a consequence, the distribution /xi : t'^ h-^ 
Et'es^ M2(t2)<5(i2>i'i) satisfies (pf/\ 
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As a consequence, for all 112 € Sat{ip^2^^), we have 
inf^ f E ^i2{t'2)6{t'2A)d{t'2A) 

< E M2(s'l,s'2,/,fc^K(si,4,/,4)^(s'l,s'2,/,fc^)^) 

+ E M2(s'i,4,/,fc^M(K,4,/,fc2)',(s'i,4,/,fci-i)') 

< E M2(si,S2,/,fc2)<(s'l,4,/,^2)',K,4,/,fcl-l)') 
A;i</C2 

< E M2(s'ii 52) /) ^2)'^'°^""^ by induction 

Since this is true for all 112 € Sat{Lp^2^^), we thus have 

Finally, we have d{t2^ ti) < AA*^^^^ = A*^, which proves the induction. 
- For all state = (sq, Sq, e, K2) G Tg^, there exists a state = (sq, Sg, e, iCi) G Tg^ 
such that d{tl,tl) < X^KAs a consequence, we have d{Ni N2, Ni N2) < 
X^K 

Let e > 0. Since A < 1, there exists K E N such that X^ < e. As a consequence, 
by the above lemma, we have that for all K < Ki < K2, 

d{Ni N2, Ni N2) < A^i < A^ < £. 

The sequence (iVi \^ N2)k is thus bi-Cauchy. Hence, because of Proposition 1, the 
sequence (of sets of PA) {\Ni \^ N2\)k is also bi-Cauchy. The other two items show 
that they converge. 

2. Theorem 2 shows that the sequence ([A^i \^ A^2])k converges in a set- theoretic 
sense (as a direct limit), and that limx-^oo I^^i ^2! = {Ni} \IN2\. Hence (|iVi] \ 
|7V2], lim^^^[Ari\^7V2] = 0, andby continuity of rf*, lim^^^ dt([7Vil\|7V2l, IA^i\-^ 
A^2l = 0. 

3. Finally, we prove that limi^^oo ^(-^i \* -^^^2: ^^'^i \^ ^^^2) = 0. This proof is very 
similar to the proof of Lemma 2 above: we can show that the distance between Nx \* N2 
and A^i \^ N2 is bounded as follows: 
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d{NiY N2,Ni\^ N2)<X^. 

Let TVi N2 = = {S'^,A,L^^,AP,V^,Tf^) and Ni \* N2 = N* = 
{S*,A,L*, AP, V* , To* ) . We start by proving by induction on 1 < fc < fsT that for all 
(si, S2, e) e Si X {S2 U ±) X U e), we have d((si, §2, e)*, (si, S2, e, k)) < X'^. The 
only difference with the proof of Lemma 2 is in the choice of the function S : S* x — )• 
[0, 1] in the induction part. Here, we choose 5 as follows: 

/i((ct' e' n (o" «" f h'W — J ^ if s'l = s'l A s'2 = S2 A f = f A k' = k - 1 
d[[s^,S2,n,[Si,S2,} jo otherwise 

The rest of the proof is identical, and we obtain that for all 1 < A; < ii' and for 

all (si, S2, e) G S'l X {S2 U ±) X U e), we have d{{si, S2, e)*, (si, S2, e, k)) < X''. 
In particular, this is also true for initial states. As a consequence, for all state = 
{si, sl,e) G To*, there exists a state = (sj, s§, e, K) e T^^ such that d{t*o, t^) < X^ . 
As a consequence, we have d{Ni \* A^2, -^1 \''^ -^2) < X^ . 
As a consequence, we obtain: 

lim d{Ni \* N2, Ni \^ N2) = 0. 

K^oo 



□ 



